Method and apparatus for multiplexed oversampled analog to digital modulation

ABSTRACT

Apparatus for time multiplexed oversampled analog to digital modulation is provided. Embodiments include modulators (60, 300 and 450) and focal plane optical imagers (100, 160 and 200). Embodiments generally include a plurality of storage elements (70) to hold modulation residues. Multiplexor and demultiplexor structures (72, 74) enable residues to be updated and stored for a plurality of modulated signals. Specific embodiments (500, 600) are shown in CMOS and mixed CMOS-CCD technologies.

This application is a continuation-in-part of U.S. application Ser. No.885,474 filed May 19, 1992 and now U.S. Pat. No. 5,248,971.

FIELD OF THE INVENTION

The present invention relates generally to analog to digital (A/D)conversion and more particularly to a multiplexed oversampled analog todigital modulator suitable for processing an array of analog inputs, asfor example, in a optical imager, to produce a digital output.

BACKGROUND OF THE INVENTION

Present approaches in focal-plane video imaging systems use some form ofanalog mutiplexing of the pixel data in order to read the imageinformation. It is this multiplexing which defines the so called videodata rates. In broadcast television, for example, the 30 hertz pixeldata rate is multiplexed to the 4 megahertz video data rate. This samesituation exists in industrial and military video systems where pixelrates are usually below 3 kilohertz and analog multiplexing is used withresulting megahertz video rates.

These multiplexing approaches have necessitated the use of analog todigital conversion processes employing high speed circuitry which, as apractical matter cannot be readily integrated with a focal-plane sensor.Moreover, the typical A/D converter in these applications comprises ahigh-speed video, flash converter which is generally considered tooexpensive for use in consumer applications.

As is discussed in Oversampling Delta-Sigma Data Converters, edited byJames C. Candy and Gabor C. Temes, IEEE Press, 1992, New York,oversampled analog to digital (A/D) converters are known which usecoarse quantization at a high sampling rate combined with negativefeedback and digital filtering to achieve increased resolution at alower sampling rate. Such converters may, therefore, exploit the speedand density advantages of modem very large scale integration (VLSI)while at the same time reducing the requirements for component accuracy.

In a type of oversampled A/D converter generally known as a delta-sigmamodulator, the analog input is sampled at a rate well above the Nyquistfrequency and fed to a quantizer via an integrator. The quantized outputis fed back and subtracted from the input. This feedback forces theaverage value of the quantized output to track the average analog inputvalue.

SUMMARY OF THE INVENTION

The present invention is directed to low cost analog to digital (A/D)conversion apparatus suitable for processing an array of analog inputsderived from an energy imager, such as an optical imager used in videocameras.

More specifically, the invention is directed to such A/D conversionapparatus which is capable of being located on or adjacent to thefocal-plane sensor of an optical imager and which is characterized bythe use of a time muliplexed oversampled conversion technique.

In accordance with a preferred embodiment, an array of analog inputs aretime multiplexed into an oversampled A/D modulator which produces, foreach input, a single-bit output that oscillates about the true value ofthe input at rates well above the Nyquist rate (the Nyquist rate beingtwice the highest signal frequency of interest). A plurality of analogstorage elements are provided, each for storing the analog residueassociated with the modulation of a different one of the analog inputs.Each analog storage element is periodically connected into the modulatorcircuit for modulation in a feedback loop to produce an updated residuevalue which is then stored in one of the analog storage elements.

The conversion apparatus thus produces at its output, a bit stream whichis representative of the multiplexed analog inputs. This output may,subsequently, be processed through a digital filter that averages itscoarse, single bit values and reduces the data rate to produce a highresolution output at the Nyquist rate.

In a preferred system embodiment, the analog inputs are generated by anarray of light sensitive (including visible, infrared and ultraviolet)elements, e.g. charge control devices (CCD), mounted coincident with thefocal plane of an image apparatus onto which an image is focused. TheA/D conversion apparatus thus produces a bit stream outputrepresentative of the light energy received by the elements of thearray. Other input signals associated with the reproduction of the image(e.g. audio, light intensity control) can be produced by a related arrayof light insensitive elements which can also be multiplexed into the bitstream.

In accordance with a further system embodiment, the integration functionassociated with the oversampled modulation and the analog storageelements for storing each analog residue are realized with an array ofintegration elements (e.g. CCD wells) arranged in close physicalrelationship with the light sensitive array.

The novel features of the invention are set forth with particularity inthe appended claims. The invention will be best understood from thefollowing description when read in conjunction with the accompanyingdrawings.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram of a delta-sigma modulator connected betweenan analog signal and a decimator;

FIG. 2A is a block diagram illustrating a preferred embodiment of amultiplexed oversampling analog to digital modulator in accordance withthe present invention;

FIG. 2B is a timing diagram pertaining to the modulator of FIG. 2A;

FIG. 3A is a block diagram of a preferred optical imager embodiment inaccordance with the present invention;

FIG. 3B is a schematic diagram of the imager of FIG. 3A;

FIG. 3C illustrates an alternate position of the switches of FIG. 3B;

FIG. 3D is a timing diagram pertaining to the modulator of FIGS. 3A, 3Band 3C;

FIG. 4A is a block diagram of another preferred optical imagerembodiment;

FIG. 4B is a schematic diagram of the imager of FIG. 4A;

FIG. 5 is a schematic diagram of another preferred optical imagerembodiment;

FIG. 6 is a block diagram of an optical imager and monitor system inaccordance with the present invention;

FIG. 7 is a block diagram of another preferred multiple input analog todigital oversampling modulator embodiment;

FIG. 8 is a schematic diagram of a typical focal plane analog readoutsystem;

FIG. 9 is a block diagram of an analog to digital modulator inaccordance with the invention;

FIG. 10A is a schematic diagram of a multiplexed oversampled analog todigital conversion system incorporating the modulator of FIG. 9;

FIG. 10B is a timing diagram of the system of FIG. 10A;

FIG. 11A is a schematic diagram of another multiplexed oversampledanalog to digital conversion system incorporating the modulator of FIG.9; and

FIG. 11B is a plan view of the CCD structure of a sensor circuit of FIG.11A.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1 is a block diagram of a prior art oversampling A/D modulator 20of a type described in Oversampling Delta-Sigma Data Converters, editedby James C. Candy and Gabor C. Temes, IEEE Press, 1992, New York. Themodulator 20 responds to an analog input from signal source 22 toproduce a bit stream output 14. The modulator 20 includes a differencer28 having a + input terminal to which the analog input is applied. Theoutput of differencer 28 is connected through an integrator 30 to an Nbit A/D converter 32 which produces the output bit stream 24. A feedbackloop 33, from the output of converter 32 to the - input terminal ofdifferencer 28, includes an N bit D/A converter 34. Configurations ofthe modulator 20 using single bit converters 32, 34; i.e. where N=1, aregenerally referred to as delta-sigma modulators. The portion of themodulator 20 comprised of converters 32 and 34 is frequently referred toas a quantizer 26.

In operation, the converter 32 produces a bit stream output 24responsive to the output 42 of integrator 30. Converter. 34 produces ananalog output 40 comprising a somewhat coarse analog estimate of theoutput 42. The output 40 is subtracted from the analog input 22 at thedifferencer 28 to form a quantizer error 44. Integrator 30 integratesthis error 44 to form an integrated quantizer error over time at itsoutput 42. The negative feedback of the loop acts to minimize theintegrated quantizer error over time so that the average value of theencoded signal representation, at the modulator output 24, is forced totrack the average analog input from source 22.

A decimator/low pass filter 50 removes the noise produced by themodulator's coarse quantization and processes the bit stream 24 toproduce, at its output 52, a finer approximation of the input signal 22at a lower rate (e.g. the Nyquist rate). As stated in the above citedreference, oversampling modulators can use simple and relativelyhigh-tolerance analog components which facilitates their realization inmodem very large scale integration (VLSI) techniques.

The present invention is based on the recognition that basic delta-sigmamodulator principles can be utilized in a time multiplexed system forprocessing an array of analog signal inputs. This recognition leads toimproved implementation of various devices, e.g. video imagers, whichcan be realized in modem integrated circuit techniques with significantcost, reliability and size advantages.

Attention is now directed to FIG. 2A which illustrates an initialembodiment of the invention in the time multiplexed analog to digitalmodulator 60. The modulator 60 includes, in an arrangement similar tothe feedback loop 33 of the modulator 20, an N bit A/D converter 62, anN bit D/A converter 64 and a differencer 66. However, the integrator ofthe modulator 60 is comprised of a secondary feedback loop 68 includinganalog memory 70, demultiplexor 72, multiplexor 74 and summer 76 whichtogether form a sampled data integrator.

A plurality of analog signals 80 are time multiplexed to the differencer66 through an input multiplexor 82 under command of a channel select andtiming circuit 84. The channel select and timing 84 can command thedemultiplexor 72 and multiplexor 74 to access, for each input signal 80,a corresponding storage location in the memory 70 (e.g. signal 80a has acorresponding storage location 70a). In a manner similar to themodulator 20, the modulator 60 output is processed through adecimator/low pass filter 86.

The concept of the modulator 60 may be addressed with reference to bothFIG. 2A and the modulator timing diagram of FIG. 2B. Through the channelselect lines 90, shown in FIG. 2A, the channel select and timing 84 can,in a repeating time sequence, command the input multiplexor 82 to directsignals 80a, 80b . . . and 80m to the differencer 66 and, in acorresponding time sequence, command the demultiplexor 72 andmultiplexor 74 to access memory locations 70a, 70b . . . and 70m. Thechannel select sequence is indicated by high channel select signalconditions 92a, 92b . . . and 92m in FIG. 2B.

During a first portion of the high signal condition 92a, the channelselect and timing 84 commands, through a read line 94, an analog residue(the integrated quantizer error of the modulator 20 in FIG. 1),presently stored in memory location 70a, to be read, via the multiplexor74, into the summer 76 and the A/D converter 62. The A/D converter 62and D/A converter 64 place a quantized estimate of this present residueat the differencer 66 and a digitally encoded representation at theinput 95 of the decimator 86. The present estimate is differenced(subtracted) at the differencer 66 from the present value of the inputsignal 80a to form a present error which is summed in the summer 76 withthe present residue to form a new analog residue.

During a remaining portion of the high signal condition 92a, the channelselect and timing 84 commands, through the write line 96, thedemultiplexor 72 to write the new analog residue into the memorylocation 70a. In the timing diagram of FIG. 2B, the read and write timeportions of the high channel select signal condition 92a are indicatedby the high 97 and low 98 conditions of the read/write select signalwhich appears on the read line 94 and write line 96 in FIG. 2A. Thisprocess for forming and writing a new analog residue is analogous to thecoarse estimate subtraction and integration of the modulator 20 of FIG.1.

This processing, including reading present stored residues from thememory 70 and writing new residues to the memory 70, is repeated foreach of the other input signals 80 during their corresponding highchannel select conditions 92b . . . 92m after which, the sequencerepeats. Thus, for each input signal, modulation results in presentanalog residues being replaced with new analog residues to maintain theintegrity of the quantizer error integration history. This enables timemultiplexed oversampled analog to digital modulation of the plurality ofanalog input signals 80.

It should be understood that FIG. 2B illustrates general residueread/write timing relationships of the modulator 60 of FIG. 2A and isnot intended to limit the order or time duration devoted by themodulator 60 to each input signal 80 to the particular sequence shown.

Oversampled modulation theory, as described in the above citedreference, indicates that selecting the value of N, for the N bit A/Dconverter 62 and the D/A converter 64, at a higher number increases thesignal-to-noise ratio (equivalently, the number of bits of resolution)achieved by the modulator 60 for a given oversampling rate. On the otherhand, selecting the value at a lower number (e.g. one) simplifies themodulator 60 structure.

FIGS. 3A and 3B respectively depict a block diagram and a schematicdiagram of another preferred embodiment, in accordance with the presentinvention, intended to be mounted coincident with the focal plane 101 ofan optical imager 100, e.g. a video camera. In contrast to the modulator60 of FIG. 2A, the imager 100 has its analog input signals generated bya light sensitive (including visible, infrared and ultraviolet) imagecollection array 102. An interline transfer array 104 then multiplexesand transfers these signals to modulators 106 associated with columns ofa residue storage array 108 (for clarity of illustration the modulators106 of FIG. 3B are shown-as a single modulator 106' in the block diagramof FIG. 3A).

In the embodiment of FIG. 3B, the interline transfer array 104 functionsto perform the m-to-1 multiplexing function of multiplexor 82 of FIG.2A. The residue storage array 108 analogously performs the 1-to-mdemultiplexing function of demultiplexor 72 and the m-to-1 multiplexingfunction of multiplexor 74. More particularly, the arrays 102, 104 and108 are each arranged in an orthogonal relationship with 1 through xcolumns (indicated at the top of the focal plane 101 for arrays 102, 104and along columns of the array 108) and m elements in each column, i.e.m rows. A modulator 106 is disposed between each column of the transferarray 104 and an associated column of the residue storage array 108.

To simplify the flow of charges through the modulators 106, the summer110 and differencer 112 of each modulator have been transposed (amathematically equivalent operation) from their corresponding positionsin the modulator 60 of FIG. 2A. In addition, columns of the memory 108are arranged in the feedback leg of the modulator feedback loop ratherthan the feedforward leg as in the modulator 60 and a separate storagewell 109 is associated with each modulator.

The arrays 102, 104 and 108 and storage well 109 are preferablyfabricated in charge control device (CCD) structures. Such structures,and methods of moving electrical charges along CCD wells thereof, arewell known in the imager art. The signals modulated in the imager 100are charge potentials collected for picture elements (pixels) of thefocal plane 101 by the collection array 102. The collection array 102 iscomprised of light sensitive collection wells 122 (also indicated by theletter C for collection), each associated with a different pixel. Lightphotons falling on the silicon gate surface of each CCD collection well122 generates a signal charge which is collected in a depletion well inthe semiconductor substrate beneath the gate.

These light induced charges are integrated into the collection wells 122over a predetermined optical frame time period (e.g. 1/30 of a second ina typical television system) after which each column of charges areparallel shifted into transfer wells 128a, 128b, . . . 128m of anadjacent transfer column. The CCD registers forming each transfer columnthen transfer the collected charge potentials serially downward to themodulators 106. The columns of the collection array 102 and theinterline transfer array 104 are arranged in an interleaved relationshipto facilitate the parallel transfer of charges therebetween. A pixelselect and timing circuit 138 controls the flow of charges along the CCDstructures of the arrays 102, 104 and 108 as required for modulation inthe modulators 106 and provides timing to the decimator/low pass filter140 for pixel identification of elements of the bit stream from thefocal plane 101.

When the modulator switches 135, 136, connected to a storage well 109and an associated storage array column 156, are in positions 135', 136'of FIG. 3B, charge potentials from a column 128 of the transfer array104 may be sequentially presented at a summer 110 along with acorresponding present residue value from an output storage element 156mof the associated column 156 of the residue storage array 108. Thesummed charge is quantized in the analog quantizer formed by the A/Dconverter 144 and D/A converter 146 and the resulting estimatedifferenced from the summed charge in the differencer 112 to form a newresidue which is moved into the input storage element 156a of the column156 of the residue storage array 108. As the quantization was performed,the A/D converter 144 digitally encoded the estimate and placed it onthe CCD register 145 to be transferred off the focal plane 101 to thedecimator 140.

After all m charge potentials from a column 128 of the transfer array104 have been sequentially processed through the corresponding modulator106 it should be apparent that the new residue values lie in the CCDwells of the residue storage array column 156 that correspond to wellsof the transfer array column 128 from which the charges weretransferred. The charge potentials that have, meanwhile, collected inthe collection array 102 during the latest frame period may now betransferred via the transfer array 104 to the modulators 106 to startanother modulation cycle.

As an alternative to the modulation cycle described above, after eachcharge potential from a column 128 of the transfer array 104 ispresented at the corresponding modulator 106 along with the presentresidue from a corresponding column 156 of the storage array 108, theswitches 135, 136 may be placed in the positions 135", 136" of FIG. 3C(a schematic of one modulator 106 and an associated residue storagearray column). While each charge potential remains presented at thesummer 110, it may be modulated a plurality of times through themodulator, each time reading a present residue value from the storagewell 109 to the summer 110 and writing the resulting new residue intothe storage well 109 from the differencer 112. The final residue of thisprocessing may then be placed in the residue storage array column 156with the switches in the positions 135', 136' of FIG. 3A which shiftsthe next present residue from the storage array column 156 to the summer110. At the same time the next charge potential from the transfer array104 is presented to the modulator 106 and the switches returned to thepositions 135", 136" to repeat the process.

This is continued until each charge potential from a column 128 of thetransfer array 104 has been modulated a plurality of times and its finalresidue stored in its corresponding well of a column 156 of the residuestorage array 108. In this manner a higher oversampling rate is achievedto increase the resolution of the digital representation of the chargepotentials.

The above described process, which enables modulation to dwell on apixel charge potential independently of collection array 102 frameperiods, may be more easily visualized with the aid of the timingdiagram of FIG. 3D which illustrates, with the aid of FIGS. 3B, 3C, aspecific charge potential modulation example. In FIG. 3D the pixelselect high signal conditions 150m, 150m-1, . . . 150a indicate thetimes during which each of the charge potentials, which were parallelshifted from the collection array 102 into the transfer array wells128m, 128m-1, . . . 128a at the end of a frame period, are presented atthe summer 110.

The signal conditions 152, 153, of the switches 135, 136 command signal,respectively indicate the times during which the switch positions 135",136" of FIG. 3C and 135', 136' of FIG. 3B are established. Finally, thehigh 154 and low 155 signal conditions of the read/write commandrespectively represent reading of a present residue through the switch136 (from the storage well 109 or associated storage array column 156)to the summer 110 and writing of a new residue from the differencer 112through the switch 135 (to the storage well 109 or associated column156).

After the charge potential originally in transfer well 128m is presentedat the summer 110 (high signal condition 150m) and the residue stored inwell 156m of the residue storage array has been written to the summer110 (read signal 154), the switches 135, 136 are moved from positions135', 136' to positions 135", 136" and the new residue written into thestorage cell 109 (write signal 155).

In this example, three more cycles of reading and writing of residuesthen follow. Prior to the last write command of these cycles (writesignal 155'), the switch command goes to the 153 condition which placesthe switches in the 135', 136' position of FIG. 3B so that the finalresidue is stored in storage well 156a. The switch command remains inthe 153 condition until the read command 154' is completed which placesthe next residue in the storage column 156 (originally in well m-1) atthe summer 110 for a modulation cycle as just described above.

Continuing with the sequence of high signal conditions 150 in FIG. 3D,it should be apparent that at the conclusion of the high 150a pixelselect signal, each charge potential shifted from a collection arraycolumn into a transfer array column 128 at the end of a frame period,has been cycled through four modulation cycles, beginning with itscorresponding residue from a residue storage array column 156, and thefinal residue placed back in the corresponding well of the storage arraycolumn 156.

It should be understood that the arrangement in FIG. 3B in which amodulator 106 is devoted to one column of the arrays 102, 104 is but oneembodiment of the invention and numerous equivalent arrangements can bedevised (e.g. several CCD array columns can be daisy chained to onemodulator). To simplify the circuits of the imager 100, the A/Dconverters 144 and the D/A converters 146 may be configured as singlebit converters.

Another preferred optical imager embodiment 160, arranged on a focalplane 161, is illustrated in the block diagram and schematic diagram of,respectively, FIGS. 4A and 4B. The imager 160 differs primarily from theimager 100 in that an interline integration array 162 performs thetransfer, storage and integration functions associated with theinterline transfer array 104, storage array 108, storage well 109 andsummer 110 of the imager 100.

In a manner similar to the imager 100, the interline integration array162 and a light sensitive collection array 164 are arranged in anorthogonal relationship with m elements in each of x columns. However,in the imager 160 the charges in the integration wells 166 are nottransferred to another site but rather are run through a modulationcycle and the resulting new residue returned to the associatedintegration well.

Thus, at the end of each succeeding frame period, the charges from acollection well 168 are added to the existing charges in a correspondingintegration well 166 rather than being shifted into an empty transferwell. Therefore, the integration array 162 serves as the modulationintegrator as well as storage for modulation residues.

The columns of the collection array 164 and integration array 162 areinterleaved by pairs to facilitate cycling the charges of theintegration array 162 through a modulation cycle. As seen in FIG. 4B,charges can thus be transferred across the top of two integrationcolumns 169, as indicated by the arrow 170, and through the remainder ofthe modulator at the bottom. In the imager 160, therefore, each pair ofintegration columns 169 and an associated modulator portion 172,containing an N bit A/D converter 174, an N bit D/A converter 176 and adifferencer 178, form each modulator (for clarity of illustration onlyone modulator portion 172' is shown in the block diagram of FIG. 4A).

As the integrated charges of an associated pair of integration arraycolumns 169 are processed through the modulator portion 172, they areeach quantized in the N bit A/D converter 174 and the N bit D/Aconverter 176 to form an estimate which is differenced, in thedifferencer 178, from the original charge to form a new residue which isreturned to the associated integration well. The processing continues atthe end of each succeeding frame period when the collected charges fromthe light sensitive CCD wells 168 are summed with the residue in eachcorresponding integration well 166 and the modulation repeated to formand store new residues. The modulation of all residues in a pair ofintegration columns 169 may be completed once each frame period or, forincreased resolution of the analog to digital conversion process, aplurality of times each frame period. The only requirement is that newresidues are returned to their corresponding integration wells 166 priorto shifting of charges from the collection array 164.

As described above for the imager 100, the number of bits of the A/Dconverter 174 and D/A converter 176 may be increased to achieve a highersignal-to-noise ratio for a given oversampling rate or decreased toachieve structural simplicity. Pixel select and timing electronics 180provides timing signals to the interline integration array 162 formoving charges along its CCD structure and to a decimator/low passfilter 182 for identification of elements of the bit stream from thefocal plane 161. The modulated bit stream from the A/D converters 174are transferred to the edge of the focal plane 161 by a CCD register184.

Another preferred imager embodiment 200 on a focal plane 201 isillustrated in the schematic diagram of FIG. 5. The imager 200 differsfrom the imager 160 in that a frame transfer/integration array 202 isspaced from a collection array 204 rather than being interleavedtherewith as in the case of the interline integration array 162 of FIG.4B. Each column 206 of the frame transfer/integration array 202 isfolded and connected at one end through a summer 208 to an associatedcollection column 210 to facilitate passing charges around the column206 for a modulation cycle.

Present residues in the frame/transfer array 206 wells are cycled andintegrated with corresponding charges shifted downward from thecollection array 204 at the end of each frame period and modulated tonew residues each time they are passed around the folded columns for amodulation cycle. The encoded bit stream from the analog to digitalconverters 212 is transferred across a CCD register 214 to the edge ofthe focal plane 201 and then to a decimator/low pass filter 216.

The focal plane 201 of the imager 200 also has a column 220 of lightinsensitive CCD wells 222 for collecting other signal inputs associatedwith the image focused on the array 204 (e.g. multichannel audio, lightintensity control). These signals are modulated and multiplexed onto theCCD output register 214 with the modulation from the light sensitivewells of the collection array 204.

Thus the focal plane 201 includes a structure of analog signalcollecting devices responsive to an energy pattern incident on the focalplane 201 wherein the energy pattern is modulated by the imager 200 intoa representative multiplexed bit stream. The energy sensitive structureis defined by a combination of light sensitive devices for receiving animage focused on the focal plane and analog signal sensitive devices forreceiving image associated signals.

FIG. 6 is a block diagram illustrating an optical imager/monitor system260 in which an image represented by a bit stream 262 from an imager264, in accordance with the present invention (e.g. optical imagerembodiments 100, 160 and 200), is monitored on a display 266. Thedisplay 266 can be any display having visible display elements that canbe driven on or off (e.g. electro-illuminescence, liquid crystal) andwhich are arranged in accordance with the picture elements of the focalscreen of the imager 264.

A pixel driver 268 decodes the bit stream 262 (opposite process of theencoding in the imager 264) and applies corresponding signals 270appropriate to the type of display elements in the display 266. A clock272 and row and column select circuits 274, 276 demultiplexes the signal270 to display elements of the display 266 in accordance with the mannerin which charge potentials of picture elements of the imager 264 focalplane were multiplexed onto the bit stream 262.

The average luminosity of each display element will be the average ofthe on and off duration of the imager digital output signal for eachpixel. Since the human eye will integrate anything changing faster than60 hertz, the modulated display element will appear to be a constantlevel (given that the imager 264 is operating at modulation rates higherthan 60 hertz). The bit stream 262 contains the original analog spectrumof light intensity at each picture element of the focal plane of theimager 264. The average pulse density at the display element over aNyquist sample interval will equal the average light intensity at thecorresponding picture element of the imager 264 focal plane to withinthe sampling resolution. In a similar manner, the bit stream 262 couldbe recorded on magnetic tape for later application to a display monitor.

The teachings of the invention may be extended to higher ordermodulation feedback loops, which are described in the above citedreference, to achieve a higher signal-to-noise ratio (higher resolutionin number of bits) for a given sampling rate. Higher order loops,however, also increase circuit complexity.

One embodiment of the invention illustrating their use is shown in theblock diagram of a multiplexed analog to digital modulator 300 of FIG.7. Compared to the modulator 60 of FIGS. 2A, 2B, the modulator 300 has asecond order feedback loop 302 wherein the quantizer estimate is fedback from the D/A converter 304 to a differencer 306. A demultiplexor310, memory 312 and multiplexor 314 are positioned in the feedforwardleg of this second loop with local feedback to the summer 316. Fortiming purposes the demultiplexor, memory and multiplexor of the firstfeedback loop have been positioned in the feedback leg.

In the description above of the optical imager 100 embodiment, it wasstated that the arrays 102, 104 and 108 and storage well 109 arepreferably fabricated in charge control device (CCD) structures. It willnow be shown that mulitplexed oversampled analog to digital modulationembodiments may be realized in other integrated circuit technologies.

FIG. 8 is a schematic diagram of a system 400 typically used in theimaging art as a focal plane analog readout. In this system, anintegrate and dump circuit 402a accumulates an optical signal from abuffered source 404a in the capacitor 406. The voltage developed acrossthe capacitor 406 is applied to the gate of an FET source follower 408.

The source follower 408 is periodically sampled in response to asampling signal 410 applied to a FET gate 412 and the sampled voltagemultiplexed onto a bus 414. The capacitor 406 is then discharged byapplying a reset signal 415 to FET 416 which clears the capacitor 406for the next charge and sample cycle. While the capacitor 406accumulates the next charge, other signal sources, for example thesource 404m, are sampled and multiplexed to the bus 414. Thus, thesystem 400 produces a multiplexed analog output 418 from a focal plane.

Multiplexed analog sampling circuits, such as the focal plane readout ofFIG. 8, can, in accordance with the invention, be replaced with themultiplexed oversampled analog to digital modulator 450 illustrated inthe block diagram of FIG. 9. In the modulator 450, an analog signal 452ais integrated by feedback of the signal through a delay 454 to be summedwith itself in a summer 456. The integrated signal (present residue) isperiodically multiplexed to an N bit A/D 460 by placing it onto a bus462 through a gate 464 in response to a read signal 465. The N bit A/D460 produces a digital bit stream output 468 representative of theintegrated signal and an N bit D/A 470, responsive to the bit stream468, places a quantized estimate thereof on the bus 472.

The quantized estimate is clocked through a gate 478 by a write signal480 to be subtracted in a differencer 482 from the present residue andsummed with the present value of the analog signal 452a to form a newanalog residue at the output of the summer 456. The integration,subtraction and gating functions comprise a circuit 490 which is alsoprovided for each of the other analog signals, e.g., circuit 490m foranalog signal 452m. Each of these circuits 490 are time multiplexed ontothe buses 462, 472 as described above for the circuit 490a.

Thus, for each analog signal 452, a circuit 490 is periodicallyconnected in a feedback loop, including the N bit A/D 460 and the N bitD/A 470, for modulation which produces a representative bit stream 468at the output of the N bit A/D and an updated residue value from asummer 456.

The multiplexed oversampled analog to digital modulator of FIG. 9 may berealized in a variety of integrated circuit technologies, e.g., chargemodulation device (CMD), bulk charge modulation device (BCMD),base-stored image sensor (BASIS), static induction transistor (SIT),lateral APS, vertical APS and double-gate floating surface transistor.Specific embodiments directed to realization in Complimentary MetalOxide Semiconductor (CMOS) technology and in mixed CCD and transistortechnologies are respectively illustrated in FIGS. 10 and 11.

FIG. 10A shows a multiplexed oversampled analog to digital conversionsystem 500 having a plurality of integration, subtraction and gatingcircuits 502a . . . 502m multiplexed to a feedback loop including an Nbit A/D 504 and an N bit D/A 506. In addition to FIG. 10A, the followingdescription of the system 500 will refer to the timing diagram of FIG.10B. In the circuit 502a, an analog signal 508a is collected onintegration capacitor 510 which is buffered by source follower 512.Channel select signals a through m are sequentially applied to thecircuits 502. During a first portion of the channel a select period,read a signal 514 goes high to switch FET 516 which connects the circuit502a via the bus 518 to the N bit A/D 504.

Therefore, the N bit A/D 504 places a bit stream at the output 522 whichis representative of the charge (present analog residue) on thecapacitor 510. In response to the bit stream, the N bit A/D 506 places avoltage Vest (quantized estimate) on the bus 520 which makes itavailable at the lower end of a subtraction capacitor 524. Thesubtraction capacitor 524 is connected to the integration capacitor 510by FET 525.

In the last portion of the channel a select period, write a signal 526goes high which causes a reference voltage Vref to be placed on the gateof FET 525. Consequently a charge equal to the quantity (Vest-Vref)/C,where C is the capacitance of the capacitor 524, is subtracted fromintegration capacitor 510 (where the FET 525 is assumed to be an idealtransistor having no voltage drop between gate and source). Prior to thewrite a pulse 526, a brief clear pulse 528 is placed on the gate 532which is in parallel with the subtraction capacitor 524. This removesall charges from the subtraction capacitor 524 to prepare it for thewrite pulse.

When the write a signal 526 and read a signal 514 are both low, theanalog signal 502a is again integrated in capacitor 510. Thus aquantized estimate of the present analog residue has been subtracted andthe present analog signal added to store a new analog residue on thecapacitor 510. In a similar manner, each of the other analog signals502m is multiplexed to the N bit A/D 504 and N bit D/A converter 506. InFIGS. 10A, 10B, it is assumed the clear signal 528 is common to eachcircuit 502 so that it is high prior to each write signal 526.

FIG. 11A illustrates an active pixel sensor (APS) 600 combining aplurality of sensor circuits 602 in a multiplexed oversampled analog todigital modulation mode. The embodiment 600 combines CCD integration andsubtraction with CMOS sampling, readout and multiplexing.

In a representative sensor circuit 602a, an analog signal produced by animage light ray 604 is integrated, during a charge cycle, in CCD wells605, 606 beneath photogates 607 and 608. The wells are defined by apotential 609 indicated in broken lines. The integrated charge of bothwells is then transferred via transfer gates 610, 611 and summed intothe CCD well 612 from where a voltage proportional to the summed chargeis available via sense gate 614 and source follower 615 (timingprecharge 616 applied through the gate of FET 617 is required to createthe well 612). Periodically, this voltage is multiplexed over a bus 618to N bit A/D 619 by a read command applied to FET 620.

As in the system 500 of FIG. 10A, the N bit A/D 619 places a bit streamrepresentative of the summed charge at an output 626. The N bit D/A 622,in response to the digital output 626, places a voltage Vest on the bus628 to enable the subtraction of a quantized estimate from the summedcharge in CCD well 612.

To effect this subtraction, the CCD wells 605, 606 are separated by apotential barrier 630 shown in FIG. 11B which is a plan view of thesensor circuit 602a CCD structure. The height of the potential barrier630 is proportional to the voltage Vest. A signal is also applied totransfer gate 611 which causes the summed charge in CCD well 612 to flowinto CCD well 605 with the excess charge flowing over the potentialbarrier into CCD well 606. For nonzero digital outputs, CCD well 605 isthen emptied into the diffusion 632 by a signal applied to the draingate 634. The subtracted (emptied) charge is, therefore, proportional tothe output voltage of the N bit A/D 622.

Therefore, a quantized estimate is subtracted and during the nextintegration cycle, the input signal 604 is integrated into CCD wells605, 606 to form a new residue charge. The formation of wells 605, 606and 612 and movement of charges therebetween as described above is wellknown in the CCD art.

From the foregoing, it should now be recognized that embodiments formultiplexed oversampled analog to digital conversion have been disclosedherein utilizing means for preserving modulation integration history foreach of an array of analog input signals (array as used herein refersgenerally to a plurality of input signals and more particularly, wherespecified, to a plurality arranged in a physical relationship, e.g.orthogonally as in the arrays 102, 104 and 108 of FIG. 3A).

The teachings of the invention enable an all digital videocamera/recorder to be manufactured with an improved image quality and atlower cost. The teachings permit all of the analog electronicsconventionally used in video and sound detection to be replaced with amonolithic on focal plane imager having a binary output.

A preferred implementation can use CCD (charge coupled device)electronics to detect and digitally process images, sound and cameracontrols for virtually noise free recording and display. The binaryoutput can be used to drive modulated fiat panel displays directly orcan be used with conventional filtering to interface raster scan analogdisplays. Other preferred embodiments can be effected in variousintegrated circuit technologies.

The preferred embodiments of the invention described herein areexemplary and numerous modifications and rearrangements can be readilyenvisioned to achieve an equivalent result, all of which are intended tobe embraced within the scope of the appended claims.

What is claimed is:
 1. Apparatus responsive to an array of m analoginputs for producing a bit stream output representative thereof, saidapparatus comprising:m storage elements, each capable of storing ananalog value; N bit A/D converter means, responsive to an applied analogvalue, for producing a bit stream output representative thereof;multiplexing means for periodically applying a residue analog value fromeach of said m storage elements to said A/D converter means, eachresidue analog value associated with a different one of said m analoginputs; N bit D/A converter means, responsive to said A/D convertermeans bit stream output, for producing quantized values each coarselyrepresentative of a residue analog value applied to said A/D convertermeans; demultiplexing means for periodically generating subtractionvalues corresponding to said quantized values; and means for summingeach said m analog input and its associated residue analog value andperiodically differencing said subtraction value associated therewith toform a new residue analog value.
 2. Apparatus of claim 1 wherein each ofsaid storage elements comprises a capacitor.
 3. Apparatus of claim 1wherein each of said storage elements comprises a CCD well.
 4. Apparatusof claim 1 wherein said N bit A/D converter means comprises a single bitA/D converter and said N bit D/A converter means comprises a single bitD/A converter.
 5. An imager responsive to an energy pattern incident ona focal plane for producing a bit stream representative thereof, saidimager comprising:an array of energy sensitive elements disposed on saidfocal plane for producing m analog signals; m storage elements, eachcapable of storing an analog value; N bit A/D converter means,responsive to an applied analog value, for producing a bit stream outputrepresentative thereof; multiplexing means for periodically applying aresidue analog value from each of said m storage elements to said A/Dconverter means, each residue analog value associated with a differentone of said m analog signals; N bit D/A converter means, responsive tosaid A/D converter means bit stream output, for producing quantizedvalues each coarsely representative of a residue analog value applied tosaid A/D converter means; demultiplexing means for periodicallygenerating subtraction values corresponding to said quantized values;and means for summing each said m analog input and its associatedresidue analog value and periodically differencing said subtractionvalue associated therewith to form a new residue analog value.
 6. Theimager of claim 5 wherein said energy sensitive elements comprise lightsensitive elements for each generating a charge potential related to theenergy of light focused thereon.
 7. The imager of claim 5 wherein saidarray of energy sensitive elements comprises an orthogonal array ofcharge control devices.
 8. The imager of claim 5 wherein said N bit A/Dconverter means comprises a single bit A/D converter and said N bit D/Aconverter means comprises a single bit D/A converter.
 9. The imager ofclaim 5 wherein each of said m storage elements is mounted proximate toa different one of said m energy sensitive elements.
 10. The imager ofclaim 5 wherein said energy collection elements comprise light sensitiveelements arranged in a two-dimensional array substantially coincidentwith said focal plane.
 11. An optical imager/monitor system forreceiving an energy pattern incident on a focal plane and reproducingsaid pattern onto a monitor wherein the monitor system is comprised of aplurality of display elements and each display element is selectivelyaddressable via driving circuitry, said system comprising:an array of menergy collection elements, each capable of producing an analog signalrelated to the amount of energy incident thereon; means for mountingeach of said m energy collection elements for collecting energy from adifferent area of said focal plane; means for storing m analog residues,each associated with a different one of said collection elements; A/Dconverter means sequentially responsive to said m analog residues forproducing an output bit stream, each related to one of said analogresidues; D/A converter means responsive to said output bit stream forrespectively producing analog quantized estimates, each related to oneof said analog residues; processing means for periodically modifying thestored value of each analog residue as a function of the analog signalproduced by the associated collection element and the analog quantizedestimate produced by said D/A converter means; and means for providingsaid output bit stream to said driving circuitry to selectively modulatecorresponding display elements.